{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "61580a32-65c5-44d5-86de-af8cb35ab1e3",
   "metadata": {},
   "source": [
    "# Lesson 8 (Optional): Llama Helper Function\n",
    "\n",
    "Here, you'll walk through the code that makes up the `llama` helper function that you've been using throughout the course, to see how it works.\n",
    "\n",
    "### Setup instructions for using the Together.AI service outside of the classroom\n",
    "- If you wish to make API calls to together.ai on your own, outside of the classroom, you can first create an account with [Together.AI](https://api.together.xyz/).\n",
    "- You'll get an API key. The sign up process is free, and together.ai offers $25 credit for new accounts.\n",
    "- After you get the key, you can set it in your own Mac/Linux environment with\n",
    "\n",
    "\n",
    "```export TOGETHER_API_KEY=<your_together_api_key>```\n",
    "or\n",
    "```echo 'export TOGETHER_API_KEY=<your_together_api_key>' >> ~/.bashrc``` \n",
    "- (on Windows, you can add it to your System Settings' Environment Variables)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3b36ded9-cd1a-4840-adfa-7e8175946d7e",
   "metadata": {
    "height": 46
   },
   "outputs": [],
   "source": [
    "# define the together.ai API url\n",
    "url = \"https://api.together.xyz/inference\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c049102f-bee4-481c-a41e-e4f8fc689144",
   "metadata": {},
   "source": [
    "### Optional: Python-dotenv\n",
    "- You can optionally set your API key in a text file and use python dot-env to load that API key.\n",
    "- Python-dotenv is helpful because it makes it easy to update your API keys by updating the text file.\n",
    "\n",
    "```\n",
    "!pip install python-dotenv\n",
    "```\n",
    "- In the root directory of your github repo or folder that contains your jupyter notebooks, create a `.env` file.\n",
    "- Open the file and set environment variables like this:\n",
    "\n",
    "```\n",
    "TOGETHER_API_KEY=\"abc123\"\n",
    "```\n",
    "\n",
    "Run the following dotenv functions, which will look for a `.env` file, retrieve the variables (like the `TOGETHER_API_KEY`, and load them as environment variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a308d95d-baa7-49f4-b16e-928a8a440488",
   "metadata": {
    "height": 63
   },
   "outputs": [],
   "source": [
    "# Set up environment if you saved the API key in a .env file\n",
    "from dotenv import load_dotenv, find_dotenv\n",
    "_ = load_dotenv(find_dotenv())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4ee915cc-48ab-4da4-8317-03de0bff5e18",
   "metadata": {},
   "source": [
    "- Whether you set the environment variable with or without the `dotenv` library, you can access environment variables using the `os` (operating system) library."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a3dde643-106f-4a23-b209-e50076a94296",
   "metadata": {
    "height": 63
   },
   "outputs": [],
   "source": [
    "# Set up the together.ai API key\n",
    "import os\n",
    "together_api_key = os.getenv('TOGETHER_API_KEY')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e0c243a9-b5b0-49b5-8bfd-f7f4674c1ac0",
   "metadata": {
    "height": 80
   },
   "outputs": [],
   "source": [
    "# Store keywords that will be passed to the API\n",
    "headers = {\n",
    "    \"Authorization\": f\"Bearer {together_api_key}\",\n",
    "    \"Content-Type\": \"application/json\"}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dc9a0dfe-e9c0-497a-bebc-24c8cc9967c1",
   "metadata": {
    "height": 46
   },
   "outputs": [],
   "source": [
    "# Choose the model to call\n",
    "model=\"togethercomputer/llama-2-7b-chat\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8c3fff18-0764-46f1-be74-1dbeefcc6ec9",
   "metadata": {
    "height": 63
   },
   "outputs": [],
   "source": [
    "prompt = \"\"\"\n",
    "Please write me a birthday card for my dear friend, Andrew.\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "24ce04fe-6f36-4004-b048-7e3c5329cf32",
   "metadata": {
    "height": 63
   },
   "outputs": [],
   "source": [
    "# Add instruction tags to the prompt\n",
    "prompt = f\"[INST]{prompt}[/INST]\"\n",
    "print(prompt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "30d34e02-4287-468d-b017-9e87ffe1d11b",
   "metadata": {
    "height": 63
   },
   "outputs": [],
   "source": [
    "# Set temperature and max_tokens\n",
    "temperature = 0.0\n",
    "max_tokens = 1024"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "087a60b7-2e39-4aa6-b313-0bcd176cfca8",
   "metadata": {
    "height": 131
   },
   "outputs": [],
   "source": [
    "data = {\n",
    "    \"model\": model,\n",
    "    \"prompt\": prompt,\n",
    "    \"temperature\": temperature,\n",
    "    \"max_tokens\": max_tokens\n",
    "}\n",
    "data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8947d405-f28e-402c-99f0-24cb4f09244c",
   "metadata": {
    "height": 80
   },
   "outputs": [],
   "source": [
    "import requests\n",
    "response = requests.post(url,\n",
    "                         headers=headers,\n",
    "                         json=data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6498574e-aa3e-4d74-8cb1-96cbb9ae0fb4",
   "metadata": {
    "height": 29
   },
   "outputs": [],
   "source": [
    "print(response)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "166f0536-6110-4868-9c8c-d4d9c55f69f9",
   "metadata": {
    "height": 29
   },
   "outputs": [],
   "source": [
    "response.json()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7b2bafc4-7d41-4553-a395-f612f6b19b98",
   "metadata": {
    "height": 29
   },
   "outputs": [],
   "source": [
    "response.json()['output']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "21538273-2cb0-43a9-9e13-907b49071f80",
   "metadata": {
    "height": 29
   },
   "outputs": [],
   "source": [
    "response.json()['output']['choices']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "00d77d89-9387-490a-9183-f1034162aea4",
   "metadata": {
    "height": 29
   },
   "outputs": [],
   "source": [
    "response.json()['output']['choices'][0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "da7983b3-3f50-431d-9e01-407dfad6d5fb",
   "metadata": {
    "height": 29
   },
   "outputs": [],
   "source": [
    "response.json()['output']['choices'][0]['text']"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c1221356-c2eb-4351-a034-7f96a95f3235",
   "metadata": {},
   "source": [
    "### Compare to the output of the llama helper function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d50bf4d7-870d-42d6-b54b-21e7394cc66a",
   "metadata": {
    "height": 80
   },
   "outputs": [],
   "source": [
    "from utils import llama\n",
    "\n",
    "# compare to the output of the helper function\n",
    "llama(prompt)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
